Automated fetal measurement from three-dimensional ultrasound data

ABSTRACT

A fetal parameter or anatomy is measured or detected from three-dimensional ultrasound data. An algorithm is machine-trained to detect fetal anatomy. Any machine training approach may be used. The machine-trained classifier is a joint classifier, such that one anatomy is detected using the ultrasound data and the detected location of another anatomy. The machine-trained classifier uses marginal space such that the location of anatomy is detected sequentially through translation, orientation and scale rather than detecting for all location parameters at once. The machine-trained classifier includes detectors for detecting from the ultrasound data at different resolutions, such as in a pyramid volume.

RELATED APPLICATIONS

The present patent document claims the benefit of the filing date under35 U.S.C. §119(e) of Provisional U.S. Patent Application Ser. No.60/977,494, filed Oct. 4, 2007, which is hereby incorporated byreference.

BACKGROUND

The present embodiments relate to medical diagnostic ultrasound imaging.In particular, fetal measurements are performed using ultrasound datarepresenting a volume.

Fetal biometric measurements represent an important factor for highquality obstetrics health care. Fetal biometric measurements are usedfor estimating the gestational age (GA) of the fetus, assessing of fetalsize, and monitoring of fetal growth and health. To perform themeasurements, a physician or sonographer manually searches for astandardized plane using 2-D ultrasound (2DUS) images. Manual searchingis cumbersome and contributes to excessive length in clinical obstetricexaminations. Long ultrasound examinations may lead to increased costs.

Three-dimensional ultrasound (3DUS) data may have increasing importancein radiology for fetal diagnosis. Compared to 2DUS, the main advantagesof 3D US may include a substantial decrease in the examination time, apossibility of post-exam data processing without requesting additionalvisits of the patient, and the ability of experts to produce 2-D viewsof the fetal anatomies in orientations that cannot be seen in common 2-Dultrasound exams. However, extensive manipulation on the part of thephysician or the sonographer may be required in order to identifystandard planes for measurements from the 3DUS data. The learning curveto understand these manipulation steps is quite large, even for expertusers. Usually, expert users find several landmarks in order to reachthe sought anatomy. For example, the standardized plane for measuringthe lateral ventricles in the fetus brain is referred to as thetransventricular plane. The user searches for the cavum septi pellucidi,frontal horn, atrium, and choroids plexus in order to identify theplane. Since the fetus is oriented in an arbitrary position in eachvolume, an expert sonographer may require several minutes to localizethe structures in a basic examination.

Some automated or semi-automated processes may be used to assist in3DUS. In the field of 3DUS, segmentation and registration of specificanatomical structures may be provided. However, segmentation andregistration merely separate data representing an already identifiedstructure.

In computer vision, there are methods for recognizing 3D objects usingrange images, but these applications are different in the sense that thesystem works with surfaces instead of actual volumes. Using 3-D magneticresonance imaging (3DMRI) data, a combination of a discriminantclassifier based on the probabilistic boosting tree (PBD) for appearanceand generative classifier based on principal components analysis (PCA)for shape, where the weights for these two terms are learnedautomatically, has been proposed. This is applied to the segmentation ofeight brain structures, where the system takes eight minutes to run.Segmentation of heart structures using 3-D computed tomography (CT) maybe based on discriminant classifiers and marginal space learning. Thesegmentation of four heart structures may be achieved in less than eightseconds. However, 3DUS data is different than CT or MRI data. Theorientation of anatomical structures in MRI and CT data is generallybetter constrained than that of 3DUS.

BRIEF SUMMARY

By way of introduction, the preferred embodiments described belowinclude methods, computer readable media and systems for measuring ordetecting a fetal parameter from three-dimensional ultrasound data. Analgorithm is machine-trained to detect fetal anatomy. Any machinetraining approach may be used. In one embodiment, the machine-trainedclassifier is a joint classifier, such that one anatomy is detectedusing the ultrasound data and the detected location of another anatomy.In one embodiment, the machine-trained classifier uses marginal spacesuch that the location of anatomy is detected sequentially throughtranslation, orientation and scale rather than detecting for alllocation parameters at once. In one embodiment, the machine-trainedclassifier includes detectors for detecting from the ultrasound data atdifferent resolutions, such as in a pyramid volume. One or more of theembodiments may be used alone or together.

In a first aspect, a method is provided for measuring a fetal parameterfrom three-dimensional ultrasound data. A machine-trained classifier isapplied to the three-dimensional ultrasound data. A first fetal anatomyis detected as a function of the applying. A value of the fetalparameter associated with the first fetal anatomy is measured anddisplayed.

In a second aspect, a computer readable storage medium has storedtherein data representing instructions executable by a programmedprocessor for measuring a fetal parameter from three-dimensionalultrasound data. The storage medium includes instructions for receivinguser input requesting automatic measurement of a fetal head anatomy, thefetal parameter being for the fetal head anatomy, detecting the fetalhead anatomy with a computer learnt classifier, the computer learntclassifier operable to detect from the three-dimensional ultrasound datawithout user indication of a position, and calculating a value for thefetal parameter.

In a third aspect, a system is provided for measuring a fetal parameterfrom three-dimensional ultrasound data. A memory is operable to storeultrasound data representing a fetal volume. A processor is operable toapply a probabilistic model to detect a first anatomy as a function ofthe ultrasound data and a position of a second anatomy. The detection ofthe first anatomy is performed sequentially for at least two ofposition, orientation and scale. A display is operable to display animage of the first anatomy, a value representing a measurement of thefirst anatomy or combinations thereof.

The present invention is defined by the following claims, and nothing inthis section should be taken as a limitation on those claims. Furtheraspects and advantages of the invention are discussed below inconjunction with the preferred embodiments and may be later claimedindependently or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

The components and the figures are not necessarily to scale, emphasisinstead being placed upon illustrating the principles of the invention.Moreover, in the figures, like reference numerals designatecorresponding parts throughout the different views.

FIG. 1 is a block diagram of one embodiment of a medical ultrasoundimaging system;

FIG. 2 is a flow chart diagram of embodiments of a method for measuringa fetal parameter from three-dimensional ultrasound data; and

FIG. 3 is a graphical representation of planes in a fetal head andcorresponding reconstruction images in one embodiment.

DETAILED DESCRIPTION OF THE DRAWINGS AND PRESENTLY PREFERRED EMBODIMENTS

In ultrasound imaging, shadows, speckle noise and other artifacts createa low contrast image with blurry edges. Even if it is easy for the humaneye to navigate and recognize structures, it is difficult to adaptcommon feature extraction techniques to 3DUS datasets. The size of theanatomical structures depends on the age of the fetus, which brings ahigh variance in the model for each structure. The position andorientation of each fetus is completely arbitrary, making it impossibleto constrain the search space in 3D.

A clinical system automatically indexes fetus structures despite thelimitations of ultrasound imaging of the fetus for biometricmeasurements. In one embodiment, a system is based on a machine-trainedmodel. In order to provide a completely automatic solution for theproblem of fetal anatomy indexing in 3DUS, a principled probabilisticmodel combines discriminative and generative classifiers with contextualinformation and sequential sampling. Contextual information orsequential sampling may be used alone or in combinations. The indexingof fetal anatomies allows the display of standard planes and associatedbiometric measurements of the fetal anatomy. In response to a user queryor other activation, anatomical structures of interest are detected.

FIG. 1 shows a medical diagnostic imaging system 10 for measuring afetal parameter and/or detecting fetal anatomy from three-dimensionalultrasound data. Fetal anatomies may be detected, allowing measurementof the anatomies and reconstruction of standard planes relative to theanatomies from ultrasound volume data.

The system 10 is a medical diagnostic ultrasound imaging system, but maybe a computer, workstation, database, server, or other system. Thesystem 10 includes a processor 12, a memory 14, a display 16, and atransducer 18. Additional, different, or fewer components may beprovided. For example, the system 10 includes a transmit beamformer,receive beamformer, B-mode detector, Doppler detector, harmonic responsedetector, contrast agent detector, scan converter, filter, combinationsthereof, or other now known or later developed medical diagnosticultrasound system components. As another example, the transducer 18 isnot provided, such as where the system 10 is a workstation for off-lineor later measurement of fetal anatomy.

The transducer 18 is a piezoelectric or capacitive device operable toconvert between acoustic and electrical energy. The transducer 18 is anarray of elements, such as a multi-dimensional or two-dimensional array.Alternatively, the transducer 18 is a wobbler for mechanical scanning inone dimension and electrical scanning in another dimension.

The system 10 uses the transducer 18 to scan a volume. Electrical and/ormechanical steering allows transmission and reception along differentscan lines in the volume. Any scan pattern may be used. In oneembodiment, the transmit beam is wide enough for reception along aplurality of scan lines. In another embodiment, a plane, collimated ordiverging transmit waveform is provided for reception along a plurality,large number, or all scan lines.

Ultrasound data representing a volume is provided in response to thescanning. The ultrasound data is beamformed, detected, and/or scanconverted. The ultrasound data may be in any format, such as polarcoordinate, Cartesian coordinate, a three-dimensional grid,two-dimensional planes in Cartesian coordinate with polar coordinatespacing between planes, or other format. The ultrasound data may be ofany type, such as B-mode, flow mode, Doppler mode, contrast agent,harmonic, or other ultrasound modes of imaging.

The memory 14 is a buffer, cache, RAM, removable media, hard drive,magnetic, optical, database, or other now known or later developedmemory. The memory 14 is a single device or group of two or moredevices. The memory 14 is shown within the system 10, but may be outsideor remote from other components of the system 10.

The memory 14 stores the ultrasound data, such as ultrasound datarepresenting a fetal volume. The fetal volume is a volume including atleast a portion of the fetal head, but other portions of a fetus may berepresented. The memory 14 stores flow (e.g., velocity, energy or both)and/or B-mode ultrasound data. Alternatively, the medical image data istransferred to the processor 12 from another device. The medical imageultrasound data is a three-dimensional data set, or a sequence of suchsets. The data represents a three-dimensional region. Any format may beused, such as voxels interpolated to a three-dimensional grid or datarepresenting parallel or non-parallel planes.

For real-time imaging, the ultrasound data bypasses the memory 14, istemporarily stored in the memory 14, or is loaded from the memory 14.Real-time imaging may allow delay of a fraction of seconds, or evenseconds, between acquisition of data and imaging with measurements. Forexample, real-time imaging is provided by generating the imagessubstantially simultaneously with the acquisition of the data byscanning. While scanning to acquire a next or subsequent set of data,images and measurements are generated for a previous set of data. Theimaging occurs during the same imaging session used to acquire the data.The amount of delay between acquisition and imaging for real-timeoperation may vary, such as a greater delay for initially locating fetalanatomies with less delay for measurements. In alternative embodiments,the ultrasound data is stored in the memory 14 from a previous imagingsession and used for measuring and/or generating a planar reconstructionwithout concurrent acquisition.

The memory 14 is additionally or alternatively a computer readablestorage medium with processing instructions. The memory 14 stores datarepresenting instructions executable by the programmed processor 12 fordetecting fetal anatomy and/or measuring a fetal parameter fromthree-dimensional ultrasound data. The instructions for implementing theprocesses, methods and/or techniques discussed herein are provided oncomputer-readable storage media or memories, such as a cache, buffer,RAM, removable media, hard drive or other computer readable storagemedia. Computer readable storage media include various types of volatileand nonvolatile storage media. The functions, acts or tasks illustratedin the figures or described herein are executed in response to one ormore sets of instructions stored in or on computer readable storagemedia. The functions, acts or tasks are independent of the particulartype of instructions set, storage media, processor or processingstrategy and may be performed by software, hardware, integratedcircuits, firmware, micro code and the like, operating alone or incombination. Likewise, processing strategies may includemultiprocessing, multitasking, parallel processing and the like. In oneembodiment, the instructions are stored on a removable media device forreading by local or remote systems. In other embodiments, theinstructions are stored in a remote location for transfer through acomputer network or over telephone lines. In yet other embodiments, theinstructions are stored within a given computer, CPU, GPU, or system.

The processor 12 is a general processor, digital signal processor,three-dimensional data processor, graphics processing unit, applicationspecific integrated circuit, field programmable gate array, digitalcircuit, analog circuit, combinations thereof, or other now known orlater developed device for processing medical image data. The processor12 is a single device, a plurality of devices, or a network. For morethan one device, parallel or sequential division of processing may beused. Different devices making up the processor 12 may perform differentfunctions, such as an automated anatomy detector and a separate devicefor performing measurements associated with the detected anatomy. In oneembodiment, the processor 12 is a control processor or other processorof a medical diagnostic imaging system, such as a medical diagnosticultrasound imaging system processor. The processor 12 operates pursuantto stored instructions to perform various acts described herein, such asobtaining data, detecting anatomy, measuring anatomy, and/or controllingimaging.

In one embodiment, the processor 12 receives acquired ultrasound dataduring or after scanning and determines locations of one or more fetalanatomies in the volume represented by the data. The processor 12performs or controls other components to perform the methods describedherein.

The processor 12 performs machine learning and/or applies amachine-learnt algorithm. For example, the processor 12 applies aprobabilistic model to detect fetal anatomy. The probabilistic model isa machine-learned classifier. Any classifier may be applied, such as amodel-based classifier or a learned classifier (e.g., classifier basedon machine learning). For learned classifiers, binary or multi-classclassifiers may be used, such as Bayesian or neural network classifiers.In one embodiment, a binary boosting classifier with a tree and cascadestructure is used. The classifier is instructions, a matrix, a learnedcode, or other software and/or hardware for distinguishing betweeninformation in a medical image.

The classifier may include a plurality of models or classifiers (e.g.,detectors) operable together or independently. For example, differentprobabilistic models are trained for different fetal anatomy. Theprobabilistic models may be joint or dependent. The location of otheranatomies is used to limit or define a search space for a currentanatomy and/or as a feature input for classification of another anatomy.For example, one anatomy, such as the fetal head or skull, is detectedfrom the ultrasound data. The position of the anatomy is determined fromthe volume represented by the ultrasound data. Another anatomy isdetected as a function of the ultrasound data and the position of theone anatomy. The other anatomy may be a cerebellum, a cisterna magna, ora lateral ventricle. In one embodiment, the cerebellum is detected basedon the location of the fetal skull, the cisterna magna is detected basedon the location of the cerebellum and/or the fetal skull, and thelateral ventricle is detected based on the location of the fetal skull,cerebellum, and/or cisterna magna. Other combinations of jointclassifiers may be used.

As another example of a plurality of classifiers being used as one,different probabilistic models are trained for translation, orientation,and scale of a given fetal anatomy. A marginal space training andapplication may provide efficient location detection. The most probabledata for a given anatomy by translation searching (shifting the searchwindow along three axes) is determined. The most probable size for agiven anatomy by scale searching (increasing and decreasing the size ofthe search window along three axes) is determined for the translatedlocations with sufficient probability. The most probable orientation byrotation searching (rotation of the search window along three axes) isdetermined for the translated and scaled locations with sufficientprobability. Other orders of translation, scale, and orientationsearching may be used in the sequence. By performing the detection forthe anatomy sequentially for at least two of position, orientation andscale, the number of computations may be reduced as compared toclassifying for each possible combination of translation, scale, andorientation. In alternative embodiments, translation, scale, andorientation searching are performed without sequentially limiting thesearch space or without marginal space searching.

In another example of a plurality of classifiers being used as one,different probabilistic models are trained for different dataresolutions. A data pyramid is provided, such as the same data set downsampled to different resolutions. Two or more versions of the data maybe provided. For example, the ultrasound data has voxels representingabout 1×1×1 mm cubes. The data is down sampled by half, providing a dataset where each voxel represents a 2×2×2 mm cube. This lower resolutiondata is down sampled by half, providing a data set where each voxelrepresents a 4×4×4 mm cube. In alternative embodiments, the voxels haveunequal sides or are not isotropic. Different probabilistic models areprovided as classifiers for each of the data sets. One classifier isapplied to the coarsest set. The results of the course set applicationare used by the classifier for the next highest resolution, such as bylimiting the search space. This repeats until the highest resolutiondata set is used. In alternative embodiments, a single data set is used.

The different classifiers for joint classification, marginal spaceclassification, and/or multiple resolution classification are the sameor different types of classifiers. The same or different types ofclassifiers may be used for the same type of classification, such asdifferent types of classifiers being used for different marginal spaceclassification (e.g., the classifier for translation is different thanthe classifier for scale).

In one embodiment, the probabilistic model is formed from a plurality ofprobabilistic boosting tree classifiers. Separate training and resultingmachine-trained classifiers are provided for each anatomy of interest.For each of these separate classifiers, separate probabilistic boostingtree classifiers are provided for each of the marginal space types. Forexample, the classifiers follow the marginal space learning protocol,providing a position detector using Haar wavelet features, a scaledetector using steerable features, and an orientation detector usingsteerable features. Separate marginal space classifiers are provided foreach resolution of data. For example, each detector (e.g., position,scale, and orientation for each anatomy) at 4 mm resolution is aprobabilistic boosting tree with 6 levels, and each node in the tree isa strong classifier with at most 20 weak classifiers. Each detector(e.g., position, scale, and orientation for each anatomy) at 2 mm is aprobabilistic boosting tree with 8 levels, and each node in the tree isa strong classifier with at most 20 weak classifiers. Each detector(e.g., position, scale, and orientation for each anatomy) at 1 mm is aprobabilistic boosting tree with 10 levels, and each node in the tree isa strong classifier with at most 20 weak classifiers. Any number ofclassifiers, nodes, levels, or other combinations may be used.

The detection algorithm implemented by the processor 12 searches throughmultiple hypotheses (window locations) to identify the hypotheses withhigh probabilities for each anatomy. Multiple hypotheses are maintainedbetween algorithm stages. Each stage, such as a translation stage, anorientation stage, and a scale stage, quickly removes false hypothesesremaining from any earlier stages. The correct or remaining hypothesespropagate to the final stage. Only one hypothesis is selected as thefinal detection result or a measurement location is detected frominformation for a combination of hypotheses (e.g., average of theremaining hypotheses after the final stage).

For application, the processor 12 calculates features forclassification. The same or different features are used forclassification in each stage. For example in a translation stage,features are calculated for each of a plurality of translated positionsof cubic regions of interest. Using a machine-trained translationclassifier, the features are used to rule out hypotheses correspondingto the translated positions, leaving a subset of remaining hypotheses.

The features are three-dimensional features. 3D data is used tocalculate the features. The window function defining the data is a cube,but may have other volume shapes. The window is translated, rotated, andscaled as part of searching for an anatomy. The same or different sizedwindows are used for different anatomies.

Any features may be used. Different types of features may be used forthe same classifier, or all of the features are of a same type for agiven classifier. In one embodiment, Haar wavelet-like and/or steerablefeatures are calculated. Haar wavelet-like features represent thedifference between different portions of a region. Any number offeatures may be used, such as tens, hundreds, or thousands. The machinelearning process may operate to determine a desired subset or set offeatures to be used for a given classification task. In one embodiment,the type of features used is gradient features. For example, the“steerable” features described by Zheng, et al. in “Fast Automatic HeartChamber Segmentation from 3D CT Data Using Marginal Space Learning andSteerable Features,” Proc. Int'l Conf. on Computer Vision, pp. 1-8,2007, are used. Other types of features may alternatively oradditionally be used.

Feature values are calculated for each hypothesis. For translationclassification at 1 mm for the cerebellum, the features are calculatedfor each of the possible translated window positions. The same features,such as the same Haar functions, are calculated for each of the possibletranslated positions. The translation classifier outputs a probabilityof a given possible position being the correct or desired anatomy basedon the feature values. If the probability is above a threshold, theassociated hypothesis is maintained. If the probability is below athreshold, the associated hypothesis is ruled out and discarded from thepool of hypotheses.

By ruling out one or more hypotheses, the number of possible positionsassociated with rotation and/or scale may be limited. For example,ruling out one hypothesis and leaving two hypotheses allows theorientation classifier to calculate features for different rotationsrelative to two different translations instead of three.

The processor 12 calculates the same or different features for each of aplurality of rotated positions associated with the remaining hypotheses.Hypotheses corresponding to the rotated positions are ruled out with anorientation classifier and as a function of the features. Afterapplication of the orientation classifier, a further subset ofhypotheses remains. The remaining hypotheses are for sufficienttranslations having at least one sufficient rotation.

The processor 12 calculates the same or different features for each of aplurality of scaled planes associated with hypotheses remaining aftertranslation and orientation testing. A scale classifier rules outhypotheses corresponding to the scaled windows as a function of thefeatures. After ruling out none, one or more hypotheses, a remaining setof hypotheses remains for the anatomy being detected. Other marginalspace orders may be used.

The remaining hypotheses for the lowest resolution data are used forclassification using higher resolution data. The marginal space processrepeats using the higher resolution data. The process is repeated untilone or more hypotheses remain after application of the classifiers forthe higher resolution data set. By sequentially ruling out hypothesesfor different marginal space and data resolution applications, thenumber of calculations for detecting a fetal anatomy may be quickly(e.g., seconds) determined using a computer.

The processor 12 calculates measurements of the detected anatomy. Anymeasurement may be made. In one embodiment, the classifier is trainedwith measurement annotations, such as caliper positions. The detectionof the anatomy provides the caliper positions as an output of theclassifier. The measurement corresponding to the caliper position isperformed, such as measuring a diameter or distance. Any now known orlater developed measurement may be used.

The display 16 is a CRT, LCD, plasma, projector, printer, or otheroutput device for showing an image. The display 16 displays an image ofthe detected anatomy, such as an image of a standard plane associatedwith the measurement of the anatomy. The data representing the volume isused for generating the image. Data from the volume dataset adjacent toor intersected by the plane defined by the location of the anatomy isused to generate a cut-plane or planar reconstruction image. Thedetected anatomy may or may not be highlighted or segmented.Alternatively or additionally, a value of the measurement is displayed.The value may be displayed in a chart, graph, and/or on an image.

FIG. 2 shows a method for measuring a fetal parameter and/or detectingfetal anatomy from three-dimensional ultrasound data. The method isimplemented by a medical diagnostic imaging system, a review station, aworkstation, a computer, a PACS station, a server, combinations thereof,or other device for image processing medical ultrasound data. Forexample, the system or computer readable media shown in FIG. 1implements the method, but other systems may be used.

The method is implemented in the order shown or a different order.Additional, different, or fewer acts may be performed. For example, act36 is optional. As another example, one or more of acts 26, 28, and 30are not performed.

The acts are performed in real-time, such as during scanning. The usermay view images of act 36 while scanning to acquire another datasetrepresenting the volume. The images may be associated with previousperformance of acts 20-36 in the same imaging session, but withdifferent volume data. For example, acts 20-36 are performed for aninitial scan and for subsequent scans during the same imaging session.Measurements and/or images of automatically detected anatomy may beprovided in seconds, such as 10 or fewer seconds.

For training and/or application, ultrasound data representing a volumecontaining the head of a fetus between 13 to 35 weeks of age is used.After 35 weeks, the ultrasound signal has difficulty penetrating thefetal skull. Alternatively, the volume represents the entire fetus oronly other portions of the fetus.

One or more sets of data are obtained. The ultrasound data correspondsto a data set interpolated to a regular 3D grid, displayed images (e.g.,detected and scan converted ultrasound data), beamformed data, detecteddata, and/or scan converted data. The ultrasound data represents avolume or 3D region of a patient. The region includes tissue, fluid orother structures. Different structures or types of structures react tothe acoustic energy differently. The shape of a structure or spatialaspect may be reflected in B-mode or harmonic data. The fetal head orother portion of the fetus is within the volume region. The datarepresents the region.

In act 20, user input is received. The user input requests automaticmeasurement or detection of fetal anatomy, such as fetal head anatomy.The measurement is a parameter associated with the fetal head anatomy.For example, measurement of the diameter of the cerebellum is requested.In alternative embodiments, the measurement and/or detection occurwithout user request, such as in response to activation of athree-dimensional fetal imaging application.

The user input is data, an electrical signal, or other informationuseable by a processor to indicate user activation. For example, theelectrical signal generated in response to user depression of a buttonor other user interface selection (e.g., pointer-based selection)indicates a user request. The context of the information shows theaspect requested (e.g., request of all available automatic fetal headmeasurements based on user selection of a menu item for such request).

In one semantic embodiment, the user input is text. A semantic keywordis input as a user query. The user may query the system using a limitedvocabulary of semantic keywords. Each keyword represents an anatomy ofinterest that the user wants to visualize and/or measure. For example,cerebellum, cisterna magna, and lateral ventricles anatomies may bedetected and measured. Once the user selects the keyword, the systemautomatically shows the standard plane of visualization and/or therespective biometric measure by implementing acts 22-36.

In act 22, a machine-trained classifier is applied to thethree-dimensional ultrasound data. The machine-trained classifier is anyone or more classifiers. The classifier may be a model or detector usingimaging processing, filtering, or other techniques. A single class orbinary classifier, collection of different classifiers, cascadedclassifiers, hierarchal classifier, multi-class classifier, model-basedclassifier, classifier based on machine learning, or combinationsthereof may be used. Multi-class classifiers include CART, K-nearestneighbors, neural network (e.g., multi-layer perception), mixturemodels, or others. A probabilistic boosting tree may be used.Error-correcting output code (ECOC) may be used.

The classifier is trained from a training data set using a computer. Anynumber of expert annotated sets of ultrasound data is used. For example,about 200 hundred ultrasound volumes representing fetal heads andincluding the cerebellum, cisterna magna and lateral ventricles areannotated. The annotation is a line, points, curves or volumesassociated with a measurement of the respective anatomy. The differentanatomies of each volume are annotated. This large number of annotationsallows use of a probabilistic boosting tree to learn relevant featuresover a large pool of 3-D Haar features and steerable features. Bothfeatures may be efficiently computed and be effective as a feature spacefor boosting classifiers. Other features may be used. Each classifieruses the data sets and annotations specific to the anatomy beingclassified.

In one embodiment, the classifier is a knowledge-based probabilisticmodel, such as marginal space learning using a hierarchical search. Adatabase of known cases is collected for machine learning, providing adatabase-driven knowledge-based approach. For training data,three-dimensional context information is preserved and guides thedetection process. Knowledge is embedded in large annotated datarepositories where expert clinicians manually indicate the anatomiesand/or measurement indicators for the anatomies. Training and detectingthe location of measurement indicators include detecting the associatedanatomy since the measurement indicator (e.g., a line representing thediameter) indicates the anatomy. The known cases may be spatiallyaligned or registered, such as by aligning the coordinate system to thefetal skull. The detectors are trained on a large number of annotated 3Dultrasound volumes. The classifier learns various feature vectors fordistinguishing between a desired anatomy and information not beingdetected. In alternative embodiments, the classifier is manuallyprogrammed.

For learning-based approaches, the classifier is taught to distinguishbased on features. For example, the probability model algorithmselectively combines features into a strong committee of weak learnersbased on Haar-like local rectangle filters whose rapid computation isenabled by the use of an integral image. Features that are relevant tothe anatomies are extracted and learned in a machine algorithm based onthe experts' annotations, resulting in a probabilistic model. A largepool of features may be extracted. The training determines the mostdeterminative features for a given classification and discardsnon-determinative features. Different combinations of features may beused for detecting different anatomies, the same anatomy at differentresolutions, and/or the same anatomy associated with differenttranslation, rotation, or scale. For example, different sequentialclassification stages utilize different features computed from the 3Dvolume data. Each classifier selects a set of discriminative featuresthat are used to distinguish the positive target from negatives. Thefeatures are selected from a large pool of features. The large pool isdetermined by a programmer or may include features systematicallydetermined.

A tree structure may be learned and may offer efficiency in bothtraining and application. Often, in the midst of boosting a multi-classclassifier, one class (or several classes) has been completely separatedfrom the remaining ones and further boosting yields no additionalimprovement in terms of the classification accuracy. For efficienttraining, a tree structure is trained. To take advantage of this fact, atree structure is trained by focusing on the remaining classes toimprove learning efficiency. Posterior probabilities or knowndistributions may be computed, such as by correlating anteriorprobabilities together.

To handle the background classes with many examples, a cascade trainingprocedure may be used. A cascade of boosted binary-class strongclassifiers may result. The cascade of classifiers provides a unifiedalgorithm able to detect and classify multiple objects while rejectingthe background classes. The cascade structure corresponds to adegenerate decision tree. Such a scenario presents an unbalanced natureof data samples. The background class has voluminous samples because alldata points not belonging to the object classes belong to the backgroundclass. Alternatively, the classifiers are sequentially trained withoutcascade.

The probabilistic boosting tree (PBT) unifies classification,recognition, and clustering into one treatment. For example, thetranslation, orientation, and scale classifiers are trained as aprobabilistic boosting tree. A probabilistic boosting tree is learnedfor each anatomy of interest. The classifier is a tree-based structurewith which the posterior probabilities of the presence of the anatomy ofinterest are calculated from given data. Each detector not only providesa binary decision for a given sample, but also a confidence valueassociated with the decision. The nodes in the tree are constructed by acombination of simple classifiers using boosting techniques, such asdisclosed by Tu, “Probabilistic Boosting-Tree: Learning DiscriminativeModels for Classification, Recognition, and Clustering,” Proc. Int'lConf. on Computer Vision, pp 1589-1596, 2005.

The classifier is trained and applied as a machine-trained joint,marginal space, and/or different resolution classifier. Any combinationof one or more of joint classification (act 26), marginal spaceclassification (act 28), and different resolution classification (act30) may be used. The resulting machine-trained classifier is applied asa detector of the cerebellum, fetal cisterna magna, fetal lateralventricles, or combinations thereof. The classifier may be trained todetect different, additional, or fewer fetal anatomies.

In act 26, a joint classifier is applied. The joint classifier includesdifferent classifiers for different fetal anatomies. For jointclassification, at least one of the anatomies is detected as a functionof the previous detection of another of the anatomies. For example, amachine-trained head detector detects the location of a center of afetal skull in a desired plane. Another machine-trained detector detectsthe location of the cerebellum, cisterna magna, lateral ventricles,combinations thereof, or other anatomy from the ultrasound data and as afunction of the location of the center of the fetal skull. Some of themachine-trained detectors have an input for the location detected fromone or more other detectors of different anatomy, so are dependent onthe output of the other machine-trained head detectors.

Joint classification uses contextual information. In one embodiment, thecontextual information is based on one or two cues: 1) global contextbased on the detection of the fetal skull, and 2) semi-local contextbased on the relative position, orientation and scale between anatomies.These cues may be modeled with generative classifiers. The fetal skullis the largest and most visible structure in 3D ultrasound. Thus, thefetal skull may be more reliably used as a reference to constrain thesearch space of other anatomies in the fetal brain.

A volume is a 3-D mapping V: R³→[0, 255]. A sub-volume window containinga particular anatomical structure is represented by a vector containingposition, size and orientation, as follows:θ_(s) =[p,σ,q]εR ⁷,  (1)where p=[x, y, z]εR³ is the three dimensional center of the sub-volume,σεR represents its size, q=[q₁, q₂, q₃]εR³ represents orientation (e.g.,represented using quaternions) and s represents a specific anatomy(here, sε{CB (center brain), CER (cerebellum), CM (cisterna magna), LV(lateral ventricles)}. FIG. 3 shows example planes and anatomies. Thesub-volume parameters of all the anatomies of interest:

$\begin{matrix}{\left\lbrack {\theta_{CER}^{*},\theta_{CM}^{*},\theta_{LV}^{*}} \right\rbrack = {\underset{\theta_{CER},\theta_{CM},\theta_{LV}}{argmax}{P\left( {\theta_{CER},\theta_{CM},\left. \theta_{LV} \middle| V \right.} \right)}}} & (2)\end{matrix}$are determined, where P(θ_(CER), θ_(CM), θ_(LV)|V) indicates aprobability measure of the anatomy parameters given the volume V. Thesearch space for this case is O((M⁷)^(L))=O(M²¹), where each dimensionis assumed to be partitioned into M values, and L=3 is the number ofanatomies to detect. An example value for M is in the order of 100.

The context may be used to prune the search space, possibly improvingthe accuracy and increasing the speed of the recognition systems. Anytype of context may be used. For example, global and semi-local contextsare used.

The global context is provided by the center of the brain (CB)structures derived from the whole skull of the fetus. CB may be thelargest and most distinctive feature in a 3D fetal ultrasound, so may befound reliably in most datasets. As context, the CB may be used toconstrain the search space for the other anatomies. Thus, equation (2)can be denoted as:

$\begin{matrix}{\left\lbrack {\theta_{CER}^{*},\theta_{CM}^{*},\theta_{LV}^{*}} \right\rbrack = {\underset{\theta_{CER},\theta_{CM},\theta_{LV}}{argmax}{\int_{\theta_{CB}}{{P\left( {\theta_{CB},\theta_{CER},\theta_{CM},\left. \theta_{LV} \middle| V \right.} \right)}{{\mathbb{d}\theta_{CB}}.}}}}} & (3)\end{matrix}$

Assuming the existence of the random variable y_(s)={−1, 1} for sε{CB,CER, CM, LV}, where y_(s)=1 indicates the presence of the anatomy s, theresult is:P(θ_(CB),θ_(CER),θ_(CM),θ_(LV) |V)=P({y _(s)=1}_(sε{CB,CERkLV,CM}|θ)_(CB) _(,θ) _(CER) _(,θ) _(CM) _(,θ) _(LV) V).  (4)

Discriminative classifiers capable of computing actual posteriorprobabilities (e.g., PBT) may be trained and applied for each anatomy.The following or other probabilities may be computed: P(y_(s)=1|θ_(s),V) for sε{CB,CER,CM,LV} (hereafter P(y_(s)=1|θ_(s), V)=P(y_(s)|θ_(s),V)). Using the Bayes rule, equation (4) is derived to:P(y _(LV) |y _(CB) ,y _(CER) ,y _(CM),θ_(CB),θ_(CER),θ_(CM),θ_(LV) ,V),P(y _(CB) ,y _(CER) ,y _(CM)|θ_(CB),θ_(CER),θ_(CM),θ_(LV) ,V)which can be further derived to:

$\frac{\begin{matrix}\begin{matrix}{{P\left( {{y_{LV}\backslash\theta_{LV}},V} \right)} \cdot} \\{{P\left( {y_{CB},y_{CER},{y_{CM}\backslash\theta_{CB}},\theta_{CER},\theta_{CM},V} \right)} \cdot}\end{matrix} \\{P\left( {{\theta_{LV}\backslash y_{CB}},y_{CER},y_{CM},\theta_{CB},\theta_{CER},\theta_{CM},V} \right)}\end{matrix}}{P\left( {{\theta_{LV}\backslash\theta_{CB}},\theta_{CER},\theta_{CM},V} \right)}.$The probability of the presence of LV based on the feature values isassumed to depend only on θ_(LV) and V, but the probability distributionof θ_(LV) depends on the detection and parameters of other anatomies.This is an assumption of parts independence but geometry dependency. Theconditional distribution of θ_(LV) given all other parameters is assumedto be a uniform distribution because there is no notion about the actualpresence of the other anatomies. Finally, equation (4) may be written asfollows:

$\begin{matrix}{{{P\left( {\theta_{CB},\theta_{CER},\theta_{CM},{\theta_{LV}\backslash V}} \right)} = {{P\left( {{y_{LV}\backslash\theta_{LV}},V} \right)}{P\left( {{y_{CM}\backslash\theta_{CM}},V} \right)}{P\left( {{y_{CER}\backslash\theta_{CER}},V} \right)}{P\left( {{y_{CB}\backslash\theta_{CB}},V} \right)}{{P\left( {{\theta_{CER}\backslash y_{CB}},\theta_{CB},V} \right)}.\mspace{14mu}{P\left( {{\theta_{CM}\backslash y_{CB}},y_{CER},\theta_{CB},\theta_{CER},V} \right)}}{P\left( {{\theta_{LV}\backslash y_{CB}},y_{CER},y_{CM},\theta_{CB},\theta_{CER},\theta_{CM},V} \right)}}},} & (5)\end{matrix}$where the first four terms are the posterior probabilities of eachanatomy, and the remaining terms account for the global and semi-localcontext. The detection probability described in equation (5) suggests asequential detection where CB is detected first, followed by CER, thenCM, and finally LV. Other orders may be used. Using context, thecomplexity of the detection may be reduced from O(M^(7L)) in itsoriginal form of equation (2) to O(L+1)×M⁷.

Additionally or alternatively, semi-local context may constrain evenmore the search space of the sought anatomy given the anatomies alreadyfound. For semi-local context, during the detection process, theparameter values of the detected anatomies are used to estimate adistribution in the parameter space for the subsequent anatomies to bedetected. For position, scale, and orientation parameters, it ispossible to determine an orthonormal matrix R_(s)εR^(3×3) with the threeaxis of the coordinate system lying in its rows using the orientationparameters. To produce scale invariant estimates of position for anatomyj given the parameters of anatomy i (i.e., θ_(i)):

$\begin{matrix}{{P_{j{}i} = {R_{i}\left( \frac{P_{j} - P_{i}}{\sigma_{j}} \right)}},} & (6)\end{matrix}$where P_(s)εR³ and σsεR are the center and scale of anatomy s,respectively. Given a training set {θ_(s)(k)}_(k=1), . . . , N, where kis an index to a training sample, a least squares optimization isformulated for the scale invariant conditional position as:

$\begin{matrix}{{\mu_{p{({j|i})}} = {\underset{P_{j|i}}{\arg\;\min}\;{J\left( P_{j|i} \right)}}},} & (7)\end{matrix}$where:

$\begin{matrix}{{{J\left( P_{j|i} \right)} = {\frac{1}{2}{\sum\limits_{k}\left( {{P_{j}(k)} - {{\sigma_{j}(k)}{R_{i}^{T}(k)}P_{j|i}}} \right)^{2}}}},} & (8)\end{matrix}$reading to:

$\begin{matrix}{\mu_{p{({j|i})}} = {\frac{1}{N}{\sum\limits_{k}{R_{i{(k)}}\left( \frac{{P_{j}(k)} - {P_{i}(k)}}{\sigma_{j}(k)} \right)}}}} & (9)\end{matrix}$Assuming a Gaussian distribution for pj|i, the position covariance maybe computed as:

$\begin{matrix}{\sum\limits_{{p{({j|i})}} =}{\frac{1}{N}{\sum\limits_{k}{\left( {{P_{j|i}(k)} - \mu_{p{({j|i})}}} \right){\left( {{P_{j|i}(k)} - \mu_{p{({j|i})}}} \right)^{T}.}}}}} & (10)\end{matrix}$

The estimation of the scale of anatomy i given the scale of anatomy j isdenoted as

$\begin{matrix}{\sigma_{j|i} = {\frac{\sigma_{j}}{\sigma_{i}}.}} & (11)\end{matrix}$Again, considering a Gaussian distribution for σ_(j)|_(i):

$\begin{matrix}{{\mu_{\sigma{({j|i})}} = {\frac{1}{N}{\sum\limits_{k}\frac{\sigma_{i}(k)}{\sigma_{i}(k)}}}}{\sum\limits_{\sigma{({j|i})}}{= {\frac{1}{N}{\sum\limits_{k}{\left( {{\sigma\left( j \middle| i \right)} - {\mu\;{\sigma\left( j \middle| i \right)}}} \right)^{2}.}}}}}} & (12)\end{matrix}$

Finally, the estimation of the orientation of anatomy i given theorientation of anatomy j is denoted asq _(j|i) =q _(i) +d _(q)(q _(j) ,q _(i)),  (13)where d_(q)(.) is a function that computes difference betweenquaternions. Considering a Gaussian distribution for q_(j|i):

$\begin{matrix}{{\mu_{q{({j|i})}} = {\frac{1}{N}{\sum\limits_{k}{d_{q}\left( {{q_{j}(k)} - {q_{i}(k)}} \right)}}}}{\sum\limits_{{q{({j|i})}} =}{\frac{1}{N}{\sum\limits_{k}{\left( {{q\left( j \middle| i \right)} - \mu_{q{({j|i})}}} \right)\left( {{q\left( j \middle| i \right)} - \mu_{q{({j|i})}}} \right)^{T}}}}}} & (14)\end{matrix}$

Given the parameter estimations above, the computation of the semi-localcontext probabilities are as follows:

$\begin{matrix}{{{P\left( \theta_{j} \middle| {\left\{ {{y_{1} = 1},\theta_{l}} \right\}_{{l = 1},\ldots\mspace{11mu},L,}V} \right)} = {g\left( {{{\left\lbrack {{\frac{1}{L}{\sum\limits_{l}{R_{l}\frac{P_{j} - P_{l}}{\sigma_{l}}}}},{\frac{1}{L}{\sum\limits_{l}{\sigma_{j}/\sigma_{l}}}},{\frac{1}{L}{\sum\limits_{l}{d_{q}\left( {q_{j},q_{l}} \right)}}}} \right\rbrack;}\left\lbrack \mu_{{p{({j|{\{ 1\}}})}}\mu_{q{({j{\{ 1\}}})}}} \right\rbrack},\sum} \right)}}{{with}\text{:}}{\sum{= \begin{bmatrix}\sum\limits_{p{({j|{\{ 1\}}})}} & 0 & 0 \\0 & \sum\limits_{\sigma{({j|{\{ l\}}})}} & 0 \\0 & 0 & \sum\limits_{q{({j|{\{ 1\}}})}}\end{bmatrix}}}{{and}\text{:}}\text{}{{{g\left( {{x;\mu},\sum} \right)} = {{\frac{1}{\left( {2\pi} \right)^{7/2}{\sum }^{1/2}}\exp} = {\frac{1}{2}\left( {x - \mu} \right)^{T}{\sum^{- 1}{\left( {x - \mu} \right)\text{)}}}}}},}} & (15)\end{matrix}$where θ_(j)=[p_(j), σ_(j), q_(j)] is the parameter for anatomy j, l isan index to the previous L detections, and[μ_(p(j|{l}))μ_(σ(j|{l}))μ_(q(j|{l}))] is computed by taking the sampleaverage of the estimations, and similarly for Σ.

With the use of semi-local context, the complexity of the detectionalgorithm is unaltered. In practice, only places where the semi-localcontext probability is above a threshold are searched. Empirically,places in the parameter space that are further than 2 times thecovariance of the estimated Gaussian may be avoided or not searched. Ingeneral, this reduces the search space at each search parameterdimension from M to M ½ (in embodiment, M≈100). As a result, in practicethe complexity of the detection is reduced from O(M^(7L)) in itsoriginal form of equation (2) to O(M⁷+L×M² ⁷ ). Consequently, the use ofsemi-local and global context information makes this approach linearlyscalable in terms of the number of brain anatomies.

In act 28, the classifier is applied as a sequence of marginalmachine-trained classifiers to the three-dimensional ultrasound data.One marginal machine-trained classifier is for translation of a givenfetal anatomy within a volume represented by the three-dimensionalultrasound data. Another marginal machine-trained classifier is forrotation of the given fetal anatomy within the volume. Yet anothermarginal machine-trained classifier is for scale of the given fetalanatomy within the volume.

The pose estimation for each fetal anatomy may involves 3-D position,orientation and scale resulting in 7 or 9 degrees of freedom (i.e., atotal of 21 or 27 degrees of freedom for all anatomies) in a typicalvolume of dimensions 250×200×150 voxels. This large dimensional searchspace makes a brute force approach not practical. Thus, to make theproblem tractable, sequential sampling and contextual information areused.

A sequence of machine-trained classifiers is learned and/or applied tothe three-dimensional ultrasound data. Anatomy detection estimates thepose parameters (i.e., position) for each anatomy. The pose parametersof a 3D rigid body may include 9 components: 3 translations (x, y; z), 3orientations (e.g., Euler angles w.r.t. for each axis), and 3 scales(one for each axis). One or more of the parameters may not be used, suchas not providing scale or only providing scale along one axis or thesame for all three axes.

Searching in a high-resolution 3D volume is prohibitive for onlineapplications or rapid determination. For example, a volume of100×100×100 voxels has 10⁶ hypotheses for translation. If combiningorientation and scale, a combinatorial hypothesis search space expandsdramatically. A limited set of hypotheses may be used based on anydesired criteria, such as relative expected positions of differentanatomy. By training a series of detectors that estimate anatomy poseparameters at a number of sequential stages, the number of calculationsmay be reduced. The stages are applied in the order of complexity as theparameter degrees of freedom increase (e.g., translation, thenorientation, and then scale), but other orders may be used. For example,scale may be adjusted only along two axes given a translation andorientation. In other embodiments, other learning with or withouthierarchical searching is used.

Sequential sampling is used to model probability distributions. Theposterior classifiers are modeled to compute P(y_(s)|θ_(s), V).Sequential sampling or marginal space learning provides efficienttraining and detection approaches for high-dimensional search parameterspaces. The original parameter space Ω is broken into subsets ofincreasing dimensionality Ω₁⊂Ω₂⊂ . . . ⊂Ω, and then classifiers aretrained for each subset. The samples for training the classifier inΩ_(n) are bootstrapped from Ω_(n-1), and the classifier in Ω₁ is trainedusing all possible samples.

In one embodiment, the following sequence is assumed: Ω₁=pεR³,Ω₂=[p_(s), σ_(s)]εR⁴, and Ω₃=Ω=[p_(s), σ_(s) q_(s)]εR⁷. The actualsearch space for training and detection in Ω_(n) is defined to bedim(Ω_(n))−dim(Ω_(n-1)), where dim(Ω_(n)) denotes the dimensionality ofthe Q_(n) space. In each subspace, a discriminative classifier istrained using the PBT algorithm (i.e., forming PBT_(n) for each Ω_(n))due to its ability of representing multi-modality distributions inbinary classification problems.

This process results in a training and detection complexity figures ofO(M³), where M is the number of quantized parameter values perdimension. This represents a reduction in terms of complexity of theoriginal algorithm in equation (2). Sequential sampling and the use ofcontextual information reduce the complexity from O(M^(7L)) toO(M³+L×M^(3/2)). This reduction allows for the detection of additionalanatomies with little impact on the overall detection complexity.

Orientation may be determined using quaternions or Euler angles. Thespace of possible orientations is usually represented with the threeEuler angles. Euler angles are easy to implement and understand, butthey have several drawbacks to represent the orientation space. First, auniform step size over possible Euler angles does not generate a uniformsampling in the space of orientations, which makes Euler anglesimpractical for uniform sampling. Second, the representation for eachorientation is not unique, which makes it difficult to define asimilarity measure between two orientations expressed in Euler angles.In other words, Euler angles are a chart of the space of orientationswith singularities (i.e., non-smooth). Consequently, two similarorientations might have very different Euler angles, which make itdifficult to compute statistics and distances. For example, if theZ×Z-convention is selected with (γ, β, α) as the three Euler angles, asingularity is provided along the line β=0. The triplet (0.7, 0.0, 0.3)gives the same orientation as the triplet (0.0, 0.0, 1.0).

The concepts on quaternions proposed for molecular modeling may be usedto represent the space of orientations in 3D. The problems above aresolved using unitary quaternions to express orientations. Eachorientation may be defined as a point in the hypersphereS³=3{pεR⁴|∥p∥₂−1} with opposite points identified. This equivalencerelation defines the space of orientations as the following quotientspace:S0(3)=S ³ /{q,−q}|q=[q ₁ ,q ₂ ,q ₃ ,q ₄ ]εR ⁴ ;∥q∥ ₂ ²=1},where the operator “/” denotes the quotient space given by theidentification of {q˜−q} in S³. Two properties from quaternions areused. First, composition of two rotations may be computed as amultiplication of two quaternions. If R(q) with qεSO(3) represents oneorientation and R(p) with pεSO(3) represents another, then R(p)∘R(q)=p·q where q is the conjugate of q¹. Second, there is a distance preservingmap between SO(3) and a ball in R³. This map allows use in SO(3) ofstandard statistical tools from R³. ¹ q=[q₁,−q₂,−q₃,−q₄]

Each quaternion may also be expressed as q=[cos(θ/2)v·sin(θ/2)]εSO(3)with vεR³ s.t. ∥v∥₂=1 and θε(−π,π). v represents the axis of rotationand θ the angle of rotation around that axis. Then, the definition ofthe distance preserving map is the following:

$\begin{matrix}{{{f\text{:}\mspace{14mu}{{SO}(3)}}->\mspace{20mu}\left. q\mapsto{u\mspace{14mu}{with}\mspace{14mu} u{}v\mspace{20mu}{and}\mspace{14mu}{}u{}} \right.} = \left( \frac{{\theta } - {\sin\left( {\theta } \right)}}{I\; I} \right)^{\frac{1}{3}}} & (16)\end{matrix}$The same way a hemisphere may be “flattened” into a disc in R²preserving geodesic distances, Equation (16) “flattens” the quotientspace into a ball in R³.

Using the two properties explained above, the orientations may bemanipulated, and statistics and metrics may be computed in the space oforientations. d_(q)(q_(j), q_(i)) in equation (13) may be defined as:dq(q _(j) ,q _(i))=∥f(q _(j))−f(q _(i))∥₂,  (17)where f is defined in equation (16).

The space of orientations may be sampled uniformly with different angleprecision. The sampling points for different resolutions may be storedin memory since the sampling points are not easy to calculate on thefly. For example, to achieve 110 accuracy, only 7416 samples are needed.Using constant step size in Euler angles 36*36*18=23328 samples areneeded. Since the complete space of orientations in 3DUS is sampled,quaternions bring calculation savings to this task.

The orientation is determined as part of the marginal space approach.The classifiers for this sequential approach are the same or different.For example, different features are used for translation, orientation,and/or scale. For the classifiers at the translation stage, Haarwavelet-like features are used, but other features may be provided. Haarwavelet-like features are calculated efficiently using integralimage-based techniques. For classifiers at the rotation and scalestages, gradient or steerable features are used, but other features maybe provided. Steerable features constitute a flexible framework, where afew points are sampled from the volume under a spatial pattern (e.g., aregular grid). A few local features are extracted for each samplingpoint, such as voxel intensity and gradient. To evaluate the steerablefeatures under a specified orientation, the sampling pattern iscontrolled, and no computationally expensive volume rotation isinvolved. The computation of steerable features does not require volumerotation and re-scaling, which are computationally expensive.

To apply the classifier, features are calculated. The features arecalculated for each of the possible anatomy positions. Other featuresmay be calculated regardless of the possible anatomy position, such aswhere a feature for a sub-volume may be determinative in combinationwith other features.

For each possible anatomy position, the features for a givenclassification are calculated. For the translation stage, the possibleanatomy positions relate to different positions translated along threeaxes. For example, Haar features are calculated for classifying whethera given translation possible anatomy position may be the desiredanatomy. For the rotation stage, the possible anatomy positions relateto rotation about the three axes at remaining translation positions. Forthe scale stage, the possible anatomy positions relate to different sizeregions at the remaining rotation and translation positions. Differentfeatures may be calculated for different stages. Different features maybe calculated for different views being detected.

In act 30, the classifier is trained and applied to the ultrasound dataat different resolutions. For example, one detector is applied to acoarse sampling of the ultrasound data. Another detector is applied to afiner sampling of the ultrasound data. The search space of the laterapplied detector is a function of an output of the earlier applieddetector. For example, the course sampling is used to determine alocation, and the search for applying a detector to a finer sampling iscentered around the location. More than two different samplings andassociated detectors may be used, such as providing three or moredifferent resolutions of data. Any order of coarse-to-fine,fine-to-coarse, or other orders may be used for sequential sampling.

For sequential sampling to provide data at different resolutions, theinitial parameter space is partitioned into sub-spaces of increasingdimensionality, where the PBT classifiers are trained sequentially ineach of these sub-spaces using bootstrap samples. A pyramid datastructure is provided. The training sets are selected for the detectorsat different levels depending on the complexity of the detection task.At the coarse level, the negative anatomy positions are far from thepositive anatomy positions and randomly sampled across reasonableconfigurations while maintaining a relatively large gap (e.g., anyempirically determined spacing) from the positives. At the fine level,negatives are selected only within an empirically determinedneighborhood of the positives in accordance to the search strategy,while decreasing the gap in between as compared to the coarse level.

The same pyramid sequence is used during the detection process. Thefeatures are calculated from the ultrasound data representing thevolume. The features are calculated from the data at differentresolutions. The sets represent the same object in the same volume.Features are calculated from a coarse set and then in a fine set of thevolume pyramid. The machine learning may determine the determinativefeatures. For each determinative feature, a data set at thecorresponding resolution is provided.

The classification using data at different resolutions is used alone orin combination with joint and/or marginal space application. Forexample, the translation, orientation, and scale for detecting thecerebellum is performed sequentially using coarse data (e.g., theoriginal data set down sampled by 4, such as 1 mm voxels down sampled torepresent 4 mm). The translation, orientation, and scale for detectingthe cerebellum are performed sequentially again using data at amid-range of sampling (e.g., the original data set down sampled by 2,such as 1 mm voxels down sampled to represent 2 mm). The translation,orientation, and scale for detecting the cerebellum are performedsequentially again using data at the highest resolution data (e.g., theoriginal data set, such as 1 mm voxels). The output from the lastdetector or classifier is one or more hypotheses for the location of thecerebellum or cerebellum measurement. This output is used as an inputfor locating the cisterna magna. The detection of the cisterna magnauses the different resolution data and marginal space detection, similarto the detection for the cerebellum, but limited or assisted by thecerebellum location.

In act 32, a fetal anatomy is detected as a function of the applying. Byapplying the machine-trained classifier, an anatomy is detected. Theanatomy may be detected in whole or in part, such as detecting thelocation of a point, line, or volume corresponding to the anatomy. Forexample, the fetal head is detected with a computer-learnt classifier.The detectors determine if a given sub-volume sample (data for apossible anatomy position) is positive or negative.

The detection may be performed in response to user activation, but isperformed without user assistance. For example, the computer-learntclassifier detects the anatomy from the three-dimensional ultrasounddata without user indication of a position. In alternative embodiments,the user confirms, adjusts, or assists in the detection.

Any fetal anatomy may be detected. For example, the fetal head,cerebellum, cisterna magna, and lateral ventricles are detected.

Each anatomy is detected independently or dependently on otherdetection. Each anatomy is detected using any number of degrees offreedom of the search window, such as translation, orientation, andscale alone each of three axes. The location searching is performedsimultaneously or sequentially. One or more sets of data representingthe same volume may be used, such as using a pyramid data structure. Inone embodiment, a joint, hierarchal, marginal space classifier is used.One or more anatomies are located, in part, based on the location ofanother anatomy. Each anatomy is located using data at differentresolutions sequentially and with marginal space sequential searching.

For joint detection, the computer learnt classifier is a jointclassifier. The joint classifier detects a fetal head anatomy as afunction of a location of other anatomy. For example, a fetal head isdetected with a first probabilistic boosting tree classifier. Thecerebellum, cisterna magna, lateral ventricle or combinations thereofare detected using other probabilistic boosting tree classifiers withthe probability distribution and/or search space limited based on thedetected fetal head and/or other anatomies.

For marginal space classification, the computer learnt classifier is amarginal space classifier. A translation detector searches fortranslation. A scale search by a scale detector is limited by a positionoutput by translation detector. An orientation search by an orientationdetector is limited by a scale output by the scale detector and/or theposition output by the translation detector. This sequential detectionmay limit complexity or increase efficiency. Possible anatomy positionsare ruled out by sequentially calculating the features for translatedpossible positions, for scaled possible positions, and for rotatedpossible positions. Each stage removes possible positions from ahypotheses list.

Any step size or search strategy may be used, such as a coarse searchwith a fine search at the locations identified as likely in the coarsesearch. The detector provides a probability for each possible position.The possible positions associated with sufficient probability aremaintained in the hypotheses pool. Sufficient probability is determinedby a threshold, by selecting the top X (where X is one or more)probabilities, or other test. The spatial distributions of probabilitiesmay be used to adjust the search, further reducing calculations.

The detectors are trained for different resolutions of the ultrasounddata in a volume pyramid. Different detectors sequentially detect usingultrasound data at different resolutions.

The detected anatomy is the possible position with the highestprobability output by the classifier. Alternatively, the detectedanatomy is the possible position with the highest average probabilityfrom the marginal space and/or data resolution stages. In otherembodiments, an average position of the remaining sufficient possiblepositions is determined. The average position is the detected anatomy.Other limitations may be used, such as averaging the position of the topY most possible positions. Alternatively, the average measurement isused.

In act 34, a value of the fetal parameter associated with the fetalanatomy is measured. The value is calculated from the ultrasound dataassociated with the anatomy and/or from spatial or temporal locationsassociated with the anatomy. Any parameter may be calculated, such asdistance, circumference, volume, change, velocity, acceleration, orother anatomy parameter. For example, a line representing the fetalanatomy is determined. The line may be a longest diameter for theanatomy in a given view. In one embodiment, the machine-trainedclassifier outputs the line. The annotated data used for training isannotated with the measurement line so that the output is the spatialline to be measured. Using the voxel size, the distance associated withthe line is calculated. The parameters are specific to a doctor,practice, or hospital. Alternatively, the biometric measurements of theanatomies are performed according to guidelines, such as the guidelinesof the International Society of Ultrasound in Obstetrics and Gynecology.

In act 36, the calculated value and/or an image are displayed. The valueis displayed as text, in a chart, or as part of a table. The value maybe labeled, such as indicating the parameter represented by the valueand the units of measurement. Other information may be displayed, suchas the gestational age derived from the value.

For an image, a planar reconstruction is created from thethree-dimensional ultrasound data. The anatomy may have a standard orpredetermined view associated with the measurement. For example, theplane includes the line used for measurement. Ultrasound datacorresponding to the view is extracted from the volume. The data is usedto generate an image for the view. Data associated with locationsintersecting each plane or adjacent to each plane is used to generate atwo-dimensional image. Data may be interpolated to provide spatialalignment to the view, or a nearest neighbor selection may be used.

Images are generated for each of the anatomies. One view may be used formultiple anatomies. All or a sub-set of the specific views aregenerated. The images may be highlighted and/or annotated to indicatethe anatomy or the measurement locations. Fewer than all available viewsmay be provided, such as displaying no more than three views and havinga priority list of views.

One example of training and application discussed for FIG. 2 isprovided. 240 volumes with expert annotations of cerebellum, cisternamagna, and lateral ventricles are collected. Other anatomies and/ornumbers of volumes may be used. The annotations are the measurementlines for each feature. The volumes have an average size of 250×200×150,but other sizes may be used. The annotation for the center of the brainuses the same annotation plane as the Cerebellum. This annotation is aline through the midline of the brain, but other planes or annotationsmay be used. The training volumes for each anatomy are obtained bybuilding a cubic sub-volume around the annotation of size k times biggerthe annotation length (e.g., k=2 for CER, k=7 for CM, k=5 LV, and k=1.5for CB). Other sub-volumes may be used.

The PBT₁, or marginal classifier for position, is trained with positivesamples formed by a box around the center location of the annotatedanatomy with fixed size and oriented according to the volume orientation(i.e., not according to the annotation orientation). The negativesamples are formed with boxes from positions δ_(p) away from this center(e.g., δ_(p)=3 voxels). The features used for PBT₁ are the 3-D Haarfeatures because of the high efficiency in computation using integralvolumes. The classifier for position and scale, PBT₂, is trained withpositive samples formed by a box around the center of the anatomy withsize proportional to the length of annotation, but oriented according tothe volume orientation. The negative samples are boxes δ_(p) away fromthe center and δ_(s) away in terms of scale (e.g., δ_(s)=2). Finally,for PBT₃, or the orientation classifier, the positive training sampleshave boxes located at the anatomy center, proportional to scale and atthe correct orientation. Negative samples are boxes δp, δ_(s), and δ_(q)away (e.g., δ_(q)=0.2). For PBT_(2,3), steerable features are usedbecause of the efficiency of their computation. The main advantage isthat, differently of the 3-D Haar features, it is not necessary toperform volume rotations to compute steerable features. The training ofPBT_(n) uses bootstrapped samples from PBT_(n-1). The process explainedabove produces the discriminative classifier P(y_(s)=1|θ_(s), V). Thesemi-local context parameters are learned generatively.

200 volumes are used for training and 40 volumes are used for testing.The training and test volumes are randomly selected, and there is nooverlap between the training and test sets. The results produced by theautomated system are compared with the results from an inter-uservariability experiment conducted with two OBGYN experts who measured theCerebellum, Cisterna Magna, and Lateral Ventricles on the same volumes.The average and standard deviation of the inter-user variability andsystem error for position, scale, and orientation are respectivelycomputed as follows:

$\begin{matrix}{{\mu_{p} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{{{p\; 1_{i}} - {p\; 2_{i}}}}}}},\mspace{14mu}{\sigma_{p}^{2} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}\left( {{{{p\; 1_{i}} - {p\; 2_{i}}}} - \mu_{p}} \right)^{2}}}},{\mu_{\sigma} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{{{\sigma\; 1_{i}} - {\sigma\; 2_{i}}}}}}},\mspace{14mu}{\sigma_{\sigma}^{2} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}\left( {{{{\sigma 1}_{i} - {\sigma\; 2_{i}}}} - \mu_{\sigma}} \right)^{2}}}},{\mu_{q} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{{d_{q}\left( {{q\; 1_{i}} - {q\; 2_{i}}} \right)}}}}},\mspace{14mu}{\sigma_{q}^{2} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{{}{d_{q}\left( {{q\; 1_{i}} - {q\; 2_{i}}} \right)}\left. {- \mu_{p}} \right)^{2}}}}}} & (18)\end{matrix}$where N is the number of volumes for testing and d_(q)(., .) is definedin equation (17). The annotations of one expert are assumed to be theground truth results. In equation (18), the index 1 denotes ground truth(i.e., one of the users) while 2 indicates either the measurements bythe other user for the case of the inter-user variability or the systemautomatic measurements for the computation of the system error. Theaverage error of the automatic results produced by the system is withinthe range of inter-user variability for all cases except for 10% to 20%of the cases for Cerebellum and Cisterna Magna position. Empirically,one tradeoff between robustness to imaging variations, noise, and posevariance and accuracy is achieved by running the system on a pyramid ofvolumes where the coarse scale is 4 mm/voxel (isotropic) and the finestscale is 2 mm/voxel. Consequently, the error results produced by thesystem have a different scale than the inter-user variability thatpartially explains this discrepancy.

This fully automatic approach may performs quickly (e.g., under 10seconds in a dual-core PC at 1.7 GHz and possibly better with increasedprocessing speed and/or code efficiency) and robustly. The systemlocates position, orientation and size of the three anatomies with anerror similar to the inter-user variability, allowing physicians andsonographers to quickly navigate through ultrasound volumes of fetalheads. A large scale semantic-based fetal structure retrieval from 3DUSmay be created, where the users type a semantic keyword and the systemreturns the structure in the volume making the 3D navigation much easierand faster.

3-D ultrasound volumes of fetal heads are automatically indexed usingsemantic keywords, which represent fetal anatomies. The automatic indexinvolves the display of the correct standard plane for visualizing therequested anatomy and the biometric measurement according to theguidelines of the International Society of Ultrasound in Obstetrics andGynecology. Anatomies are retrieved in ultrasound volumes based onsemantic keywords or with or without other input. Tens of brainanatomies may be determined, even given the small, noisy, indistinct,and/or difficult to locate appearance. The principled probabilisticmodel combines the use of discriminative/generative classifiers withglobal and semi-local context.

While the invention has been described above by reference to variousembodiments, it should be understood that many changes and modificationscan be made without departing from the scope of the invention. It istherefore intended that the foregoing detailed description be regardedas illustrative rather than limiting, and that it be understood that itis the following claims, including all equivalents, that are intended todefine the spirit and scope of this invention.

We claim:
 1. A method for measuring a fetal parameter fromthree-dimensional ultrasound data, the method comprising: obtaining thethree-dimensional ultrasound data representing a patient; applying amachine-trained classifier to the three-dimensional ultrasound data, themachine-trained classifier comprising a combination of a jointclassifier for a plurality of fetal anatomies, including a first fetalanatomy, the first fetal anatomy being detected as a function ofdetection of a location of another fetal anatomy, and a sequence ofmarginal machine-trained classifiers to the three-dimensional ultrasounddata, a first of the marginal machine-trained classifiers fortranslation of the first fetal anatomy within a volume represented bythe three-dimensional ultrasound data, a second of the marginalmachine-trained classifiers for rotation of the first fetal anatomywithin the volume, and a third of the marginal machine-trainedclassifiers for scale of the first fetal anatomy within the volume, thethird marginal machine-trained classifier for the scale corresponding todetermining a size of the first fetal anatomy within the volume;detecting the first fetal anatomy as a function of the applying; whereineach stage in the sequence of marginal machine-trained classifierssearches through multiple locations in the three-dimensional ultrasounddata to identify the locations with higher probabilities for thelocation of fetal anatomies and further removes false locationsremaining from a previous stage in the sequence of marginal machinetrained classifiers; wherein each of the fetal anatomies are detected byapplying any numbers of degrees of freedom in the marginalmachine-trained classifiers for translation, rotation and scaling;measuring the fetal parameter associated with the first fetal anatomy,the measuring providing a value of the fetal parameter; and displayingthe value.
 2. The method of claim 1 further comprising displaying aplanar reconstruction from the three-dimensional ultrasound data, theplanar reconstruction showing the fetal anatomy.
 3. The method of claim1 wherein applying comprises applying the machine-trained classifier asa detector of the fetal cerebellum, fetal cisterna magna, fetal lateralventricles, or combinations thereof.
 4. The method of claim 1 whereinapplying the joint classifier comprises applying a machine-trained headdetector, and then applying a machine-trained detector of the firstfetal anatomy, the machine-trained detector of the first fetal anatomydependent on an output of the machine-trained head detector.
 5. Themethod of claim 1 wherein applying the machine-trained classifiercomprises applying a first detector to first data comprising a coarsesampling of the ultrasound data and applying a second detector to seconddata comprising a finer sampling of the same ultrasound data, the seconddetector searching for the first fetal anatomy in the ultrasound data, asearch space of less than all the ultrasound data by the second detectorbeing a function of an output of the first detector.
 6. The method ofclaim 1 wherein applying the machine-trained classifier comprisesapplying a probabilistic boosting tree.
 7. The method of claim 1 whereinmeasuring comprises determining a line representing the first fetalanatomy as an output of the machine-trained classifier, determining avoxel size associated with the ultrasound data, and calculating adistance of the first fetal anatomy as a function of the line and thevoxel size.
 8. In a non-transitory computer readable storage mediumhaving stored therein data representing instructions executable by aprogrammed processor for measuring a fetal parameter fromthree-dimensional ultrasound data, the storage medium comprisinginstructions for: obtaining the three-dimensional ultrasound datarepresenting a patient; receiving user input requesting automaticmeasurement of a fetal head anatomy, the fetal parameter being for thefetal head anatomy; detecting the fetal head anatomy with a computerlearnt classifier, the computer learnt classifier operable to detectfrom the three-dimensional ultrasound data without user indication of aposition, the computer learnt classifier comprising both a jointclassifier such that the fetal head anatomy is detected as a function ofa location of other anatomy and a marginal space classifier includingclassification of a size of the fetal head anatomy, wherein each stagein a sequence of the marginal machine-trained classifier searchesthrough multiple locations in the three-dimensional ultrasound data toidentify locations with higher probabilities for the location of thefetal head anatomy and further removes false locations remaining from aprevious stage in the sequence of the marginal machine trainedclassifier, wherein the fetal anatomy is detected by applying anynumbers of degrees of freedom in the marginal machine-trained classifierfor translation, rotation and scaling; and calculating a value for thefetal parameter.
 9. The non-transitory computer readable storage mediumof claim 8 wherein detecting comprises detecting a fetal head with afirst probabilistic boosting tree classifier, and detecting acerebellum, cisterna magna, lateral ventricle or combinations thereof asthe fetal head anatomy using a second probabilistic boosting treeclassifier.
 10. The non-transitory computer readable storage medium ofclaim 8 wherein detecting with the marginal space classifier comprisesusing a translation search with a first detector of the computer learntclassifier, detecting using a scale search limited by a position outputby the first detector with a second detector of the computer learntclassifier, and detecting using an orientation search limited by a scaleoutput by the second detector and the position with a third detector ofthe computer learnt classifier.
 11. The non-transitory computer readablestorage medium of claim 8 wherein detecting comprises detecting withdetectors trained for different resolutions of the ultrasound data in avolume pyramid.
 12. The non-transitory computer readable storage mediumof claim 8 wherein detecting comprises detecting a fetal head,cerebellum, cisterna magna, and lateral ventricles with a joint,hierarchal, marginal space classifier.
 13. A system for measuring afetal parameter from three-dimensional ultrasound data, the systemcomprising: a memory operable to store ultrasound data representing afetal volume; a processor operable to apply a probabilistic model todetect a first anatomy as a function of the ultrasound data and as afunction of a position of a second anatomy, the detection of the firstanatomy being performed sequentially in a sequence of stages of positionof the first anatomy, orientation of the first anatomy, and scale of thefirst anatomy, the scale comprising a size of the first anatomy in thefetal volume, wherein each stage in the sequence searches throughmultiple locations in the ultrasound data to identify locations withhigher probabilities for a location of the first anatomy and furtherremoves false locations remaining from a previous stage in the sequence,wherein the first anatomy is detected by applying any numbers of degreesof freedom in the probabilistic model for position, orientation andscale; and a display operable to display an image of the first anatomy,a value representing a measurement of the first anatomy or combinationsthereof.
 14. The system of claim 13 wherein the probabilistic modelcomprises a machine learned classifier.
 15. The system of claim 14wherein the second anatomy comprises a fetal skull, the first anatomycomprises a cerebellum, a cisterna magna, or a lateral ventricle.